A Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation

Multi-layer feedforward neural networks with sigmoidal activation functions have been termed "universal function approximators". Although these types of networks can approximate any continuous function to a desired degree of accuracy, this approximation may require an inordinate number of hidden nodes and is only accurate over a finite interval. These short comings are due to the standard multi-layer perceptron's (MLP) architecture not being well suited to unbounded non-linear function approximation. A new architecture incorporating a logarithmic hidden layer proves to be superior to the standard MLP for unbounded non-linear function approximation. This architecture uses a percentage error objective function and a gradient descent training algorithm. Non-linear function approximation examples are uses to show the increased accuracy of this new architecture over both the standard MLP and the logarithmically transformed MLP.